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ABSTRACT 

This report forms the background for some current 
efforts to develop computer-based scoring systems for One-Word 
Sentence Completion data. A general discussion of response grouping 
problems and a three-way frequency table for data reduction are 
included. In addition, six ''structural” response variables are 
defined. The report concludes with an exploratory analysis of sex 
differences on the 26 variables scored by the report generator, and a 
series of intercorrelation analyses of the internal relationships 
among various parts of the system. (Author/DLG) 
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This report originated from ;i project supported by the lixcellence 
l ; und of the llniversitv of Texas and from the activities of the Computer 
Analysis of Personality project (NIMI1 06823, 1962-1967) at the University of 
Texas. It forms the background for some of the current efforts to develop 
computer-based scoring systems for One- Wo rd S en fe n ce Comp let ion (OWSC) data 
by the staff of the Rii I) Center for Teacher education. 

The report begins with a general discussion oi the problem of re- 
sponse grouping, using, a sample of the 2821 freshmen to exemplify the pro- 
cesses of data reduction. The final result of those procedures was a system 
of reducing OWSC data to a three-way frequency table: four groups of stems, 

six response themes, and throe levels of nl lect/evaluation. In addi tion, six 
"structural" response variables are defined. . 

A report generator program is described which automatical ly yields 
a two page summary description of any OWSC protocol input to .it. An example 
of the output, of this routine is included. The report concludes with nil ex- 
ploratory analyses of sox differences on the 26 variables scored by the lcpoit 
Generator, and a series of ini or correlation analyses ol the internal relation- 
ships among various parts of the system. 

Although never tit*, i I i zed in practical screening or counseling pro- 
grams, the system serves as tin experimental model of many of the processes 
to bo implemented by later development projects now under way. 
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Computer Analysis of Personality 
January, 1967 

SCORING STRUCTURE AND REPORT GENERATOR FOR THE 36-ITEM OWSC FORM 

Donald J. Veld/ran 

The central problem in the development of an objective scoring 
system for a sentence-completion instrument such as the 36-item form used 
in the Study of Freshmen is that of defining a reasonable procedure for 
reducing the variety of data to a snail set of categories useful for pur- 
poses of psychological assessment. The procedure to be described in this 
report nay be considered as an application of a four-level conceptual 
scheme originally described elsev7here (Veldman, 1967). This structure 
concerns levels at which tvo wor^Js (responses) nay be considered the same: 

(1) literal (love, love) A letter-by-letter correspondence 
establishes the identity of two responses. 

(2) syntactic (love, loved) Grammatical variants of the same 
word root are considered equivalent. 

(3) semantic (love, adore) Synonyms arising from different 
response roots are considered to have the same meaning. 

(4) pragmatic (love, happy) Words are clustered at this level 
in terms of some particular superordinate concept, such as "affective state' 
for the example. The term "pragmatic" derives from the fact that the 
cluster is defined to serve some particular purpose of the classifier. 

These four levels of classification describe rather well the se- 
quence of procedures followed by the author in dealing with the raw data 
supplied by the sample of 2321 freshmen. 

This report will describe one of many possible data-reduction 
and scoring systems which could be devieed for the OWSC instrument. As 
noted in the discussion at the end of this paper, the reduction process 
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used here was perhaps carried farther than that which would be optimum for 
development of a flexible "score compiler" system. Still, the results of 
this procedure may have heuristic value for later development of a mere 
general scoring system. 

The Need for Response Grouping 

The 2321 subjects responded to the 36-item sentence-completion 
forms with a total of 83,556 answers. Of these, 1352 (1.6%) were blanks, 
leaving a total of 82,204 actual response words to classify. When lists 
were compiled separately for each of the 36 stems, 13,743 different response 
words were obtained — a reduction to 16% of the gross total. Of these 
words, 7971 (58%) were unique responses to the stems which elicited them, 
and accounted for 9.5% of the gross total. A large proportion of these 
"unique" responses were only graiucatical variants or common synonyms for 
other more corrmon responses. Even if blanks and unique responses are not 
considered, a total of 5772 individual words remain to be coded for use in 
an objective scoring system. 

As described elsewhere (Peck, Menaker, and Veldman, 1966), Shirley 
Menaker attacked this problem in a direct fashion, although in a different 
context, using a 90-item form and a smaller subject sample. She assigned 
weights to a total of 4366 response words in order to permit automatic 
scoring of 25 different constructs. By assuming that words not appearing 
in this "dictionary" should be neutrally weighted, it was possible to avoid 
inclusion of an additional 2776 responses. In the scoring process which 
employed this weighted dictionary, all unique responses were also considered 
neutral in their implications for the 25 constructs concerned. This 
system has been demonstrated to be as valid and reliable as subjective 
scoring of the same protocols for the same constructs by individual raters. 
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A number of logical weaknesses are inherent in this system, however, as 
Dr. Menaker noted in her initial report. 

First of all, the neutral weighting of unique responses probably 
does not bias the scores systematically . On the other hand, our experience 
with these data suggests that unique responses tend to be especially signi- 
ficant in that they often reveal unusually strong feelings or prepotent con- 
cerns on the part of the subjects. Thus, if coded, they would tend not to 
have neutral weights. (At the other extreme, popular responses perhaps 
should be coded neutral in all cases.) By leaving uniques out of the system, 
we are probably ignoring some powerful information. 

This first problem is compounded when one considers the fact that 
some stems (particularly those with a few very popular responses) yield an 
average score considerably above the original neutral level. Thus, a person 
who responds with an uncommon synonym for a common response is implicitly 
given a below-average rating on the variable concerned. 

Another consideration is the rather unreasonable amount of time 
and effort required of the clinician who assigns weights to the responses 
in the dictionary — and ideally this should be done independently by each 
member of a conmittee and then the differences reconciled by concensus. 

Dr. Menaker estimates that she spent 100 hours deciding on the weighting of 
the 7142 responses for the 25 variables to be scored. For any new constructs, 
the entire list would have to be re-evaluated. For a new sentence-completion 
form, the entire process would have to be repeated for each new stem. 

Another weakness implicit in the direct response-weight procedure 
appears when the system is applied to a new sample of subjects — particu- 
larly if their verbal habits differ from those of the original sample: A 
somewhat larger proportion of the responses will not appear in the dictionary 
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— and hence the variance of the construct scores my be severely restricted 
The procedures to be described in the following section of this 
report were undertaken in order to obtain a "dictionary” with greater 
generality than a simple compilation of responses. By deriving a final 
list of response-word roots, it becomes possible to pick up a substantial 
proportion of the unique responses. Also, the root list is only 30% as 
long as the correspo ..;irg list of non-unique responses which would otherwise 
have to be coded for scoring the sample of 2321 freshmen. 

Stage One (literal equivalence ) 

As described in an earlier memorandum (Veldman, 1965), the 83,556 
individual responses of the 2321 freshmen were compiled to fom 36 lists 
containing a total of 5772 non— unique words. Further compilation of these 
36 lists resulted in a total of 2686 response words which occurred at least 
once in response to one or more of the 36 stems. These words were punched 
into separate IBM cai'ds to facilitate later processing. 

Stage Two (syntactic equivalence ) 

Grammatical variants were eliminated by reducing responses to their 
roots, after the list of 2686 words had been alphabetically ordered. Thus, 
LOVE, LOVING, LOVED, LOVES, LOVER, LOVERS, LOVINGNESS, LOVABLE, AND LOVELESS 
were replaced by the single root LOV. The negation LOVELESS (as well as all 
variants with the prefixes UN- , NO- , NOT- , and NON- ) is handled by a program- 
ed procedure when the list of roots is searched. Tlie word LOVELY was not 
removed from the list in order tint it might be separately e valuat ed. The 
particular assumptions implicit in the construction of the root list are 
open to argument, as well as to relatively easy adjustment if necessary. 

A total of 1726 roots were retained, which picked up all but 2397 (2.9%) of 
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the original 83,556 responses. . 

At this point it would have been possible to re-compile lists of 
occurrences of these roots for each stem, and to weight them for each of a 
series of constructs, yielding a scoring dictionary comparable to that de- 
veloped by Dr. Menakcr. For a number of reasons, we chose to continue the 
data reduction process tlirough the levels of semantic and pragmatic equiv- 
alence to yield a highly abstracted response-classification system. 

Stage Three (semantic equivalence ) 

The equating of roots carried out at this stage is even more sub- 
ject to debate than was the reduction at stage two. In view of the final 
goal of the present process, most of these arguments would be academic in 
this case. However, it should be noted that construct weighting could be 
carried out using the synonym groups resulting from this process. Since 
there were roughly only 450 such groups, considerable effort on the part of 
the weighter(s) would be saved — if they could agree on the synonym 
grouping. Each synonym group was assigned an "affective weight": 1 = nega- 

tive, 2 - neutral, 3 = positive. 

i > ' 

Stage Four (pragmatic equivalence ) 

.This was undoubtedly the most difficult (conceptually) and time- 
consuming (because of false starts) of the four stages. The term "pragmatic" 
is apt, since the defense of any classification scheme must lie in the uses 
for which it is intended, and its ability to serve such purposes when tested 
empirically. A variety of considerations bore upon the decisions regarding 
this choice of six general categories or "themes": 

1. Physical (body, quantity, nature, animals) 

2. Poles (people, occupations, and institutions) 



3. ftnotinnal. (affective states and attitudes) 

4. Social (interpersonal behaviors) 

5. Cognitive (ability, perception, academic) 

6. Performance (success, ambition, control) 

Since the final .response classification system includes separate 
evaluation of affective tone, this aspect of the response roots was not 
considered in evolving the six categories. Any of these six categories 
could easily be subdivided into more honogeneous sets of roots, but at an 
unknown cost due to the decreased frequencies. The significance of this 
factor will become more apparent when the full design of the protocol re- 
duction scheme is discussed in the following section of this report. 

Scoring Structure for the 36-item OWSC Form 

As mentioned previously, responses are to be automatically reduced 
to root forms and at the same time identified with regard to theme and affect 
level. The 36 sentence-completion stems were also grouped into four sets: 

1. Self Description 

2. Future Orientation and Goals 

3. Stressors and Reactions 

4. Social Attitudes and Interaction 

The particular stems assigned to each of these categories are 
shown in the printout of the individual subject report which appears later 
in this paper. The reader should note that each of these four categories 
could have been further split into two subsets; this was deemed inadvisable 

for the same reasons that only six thematic categories were chosen for the 
present system. 

A three-dimensional model thus serves to describe the current 



scoring structure for the 36-item sentence-completion form: 4 stem sets, 

6 response themes, and 3 affect levels. Theoretically, one could accumulate 
frequency distributions for each of the 72 cells in this model, and could 
then attempt to characterize a given subject or particular criterion groups 
in terms of their 72-element patterns. We may eventually attempt to assess 
the empirical validity of such a scoring system for particular external 
criteria, but the present report will focus upon a smaller set of summary 
scores derived from this 72 cell table and from the raw data of the protocol 

A set of 26 variables was defined for use in developing a report 
generator for individual protocols, as well as for use in r omparison of 
criterion groups. The first six scares reflect protocol characteristics 
which are independent of the three-dimensional scoring procedure just de- 
scribed, while the remaining 20 variables are sunrmary scores developed from 
this framework. 

Variable 1. Number of Blank Responses . Blanks are not entered 
into the content-scoring framework in order to avoid any assumptions about 
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their significance. The count, however, is reported for its value as a 
"tempering" variable. 

Variable 2. Number of Proper Name Responses . Non-public proper 
names are punched "PN," and a count of these responses is made. ("John" 
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would be coded PN, but "Kennedy" would not.) These responses are coded 
into the content system (Role Theme, Neutral Affect). 

Variable 3. Number of Popular Responses . A table of all specific 
response words which were given by more than 10% of the sample of 2321 
students is stored in the computer memory and referenced to obtain this 
score. Original response words are used in this case, as is true of all 
of the first six scores. 



Variable 4. Number of Unique Responses . As the original re- 
sponses are submitted to the root-list search system, a count is kept of 
the number of words which cannot be equated to any root in the dictionary. 
Such responses, like blanks, are not entered into the content-scoring 
framework for variables 7^26. 

Variable 5. Number of Repeated Responses . The original list of 
36 responses is examined for exact repetitions within the list. Blanks 
are ignored. Thus, a protocol where the word "argue" appears twice, and 
the word "happy" appears three times, would obtain a repetition score of 
three . 

Variable 6. Average Response Length . Each response space in 
the computer is a maximum of 10 characters. The number of characters of 
each response is determined, summed, and divided by the number of valid 
responses — since blank and proper-names are excluded in this calculation. 

Affect Analysis 

The next four variables are based on the affect level codes of 
the responses which have been retained for content analysis. (Blanks and 
uniques are excluded.) 

Variable 7. Number of Negative Affect Responses , (code = 1) 

This is a simple frequency count of responses whose roots have been assigned 
this affect code. 

Variable 8. Number of Neutral Affect Responses , (code = 2) 

Variable 9. Number of Positive Affect Responses , (code = 3) 

Variable 10. Average Affect Level . This score is a composite 
of the information represented in variables 7, 8, and 9. It is the simple 
average of the affect weights of all non-blank, non-unique responses. 
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Stem-Sot Analysis 

The next four variables register the average affect levels of the 
responses to the stems in each of the four groups of sentence-completion 
items of the 36-item OWSC fonn. Blanks and uniques are excluded from these 
computations . 

Variable 11. Self Description Affect Level . This is an average 
of the affect weights for the 10 (or less) valid responses to the steins in 
this category. 

Variable 12. Future Orientation and Goals Affect Level . This 
group is made up of nine steins. 

Variable 13, Stressors and Reactions Affect Level . This group 
contains eight stems. The nature of these stems would lead to an expec- 
tation of a relatively low affect level in the typical protocol. 

Variable 14. Social Attitudes and Interaction Affect Level . 

This group contains nine stems. 

Thematic Analysis 

The last 12 variables describe the frequency and the average affect 
level of responses whose roots had been preclassified into each of the six 
thematic categories. For any category, a zero frequency will automatically 
result in an affect level score of zero, which serves as a signal for ig- 
noring the score in statistical analyses. 



noted earlier, bodily characteristics, animals and general quantitative 



Variable 15. Frequency of P hysical Theme Responses . 

Variable 21. Average Affect of Physical Theme Responses . As 



descriptors are included in this category. 

Variable 16. Frequency of Role Theme Responses 





i 




10 

Variable 22. Average Affect of Role Thame Responses * This 
category includes persons, occupations, and references to social institu- 
tions such as marriage . 

Variable 17, Frequency of Bnotional Theme Responses * 

Variable 23. Average Affect of Bnotional Theme Responses. Both 
positive and negative affective states are included in this category. The 
major foci are love-hate, happy-sad, and tense-relaxed. ' 

Variable 18. Frequency of Social Theme Responses . 

Variable 24. Average Affect of Social Theme Responses . These 
terms are descriptive of interpersonal behaviors, as opposed to emotional 
reactions, and reflect social morality as well as social involvement. 

Variable 19. Frequency of Cognitive Theme Responses . 

Variable 25. Average Affect of Cognitive Theme Responses . 
Perceptual traits and behaviors, intellectual ability and activity, academic 
references, and generalized certainty are all included within this category. 

Variable 20. Frequency of Performance Theme Responses . 

Variable 26. Average Affect of Performance Theme Responses . 

The whole range of achievement orientation, activity, and outcomes are 
included, along with terms concerning behavioral control and generalized 
evaluation. 

Report Generator for Single Protocols . 

The scoring system described above was designed to serve as the 
basis for descriptive reports on individual protocols, as well as for . 
studies of differences among criterion groups of subjects. In its present 
form, the system will serve the latter purpose quite adequately, tut is 
only at an intermediate stage of development with respect to its applica- 
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tions to individuals. A next step would be the compilation of normative 
expectations for each of the 26 variables. Such tables could be studied 
in order to establish a network of verbal statements to be output auto- 
matically in the form of a personality description after entry of the 36 
sentence completion responses from a single protocol. An intermediate 
version of a single-protocol report generator has been programed, and a 
sample of its output is shown on following pages. The first page of the 
report re-organizes the 36 sentence completion items into the four major 
stem-groups. The stems are printed (preceded by their serial numbers in 

the original form) with the particular subject’s response words embedded. 

• 

These are bracketed by asterisks. Although no such responses occurred in 
the example protocol, a blank would appear as *NR*, while a proper name 
would be printed *PN*, since that is the way they were coded when the orig- 
inal data cards were punched. 

At the right side of this first page appear the results of the 
root-search procedure which is applied to each response in turn. The entry 
following the letter "R" is the root in the dictionary (if any) which was 
equated to each response. The term (UNIQUE) is printed when this search 

• • • » c 

was unsuccessful — as was the case with DISHEARTEN on stem 1. Note that 

negative prefixes such as UN attached to comnon roots are recognized as : j 

* . ■ ’ ■ 4 . 

such ~ as in stem 22 (third group). The negative sign is added to signify 
this kind of response. 

The term following the letter ”T" is the theme code for the 
selected root, which is left blank for NR or unique responses. The number . • • ’ 




following the letter ’’A” signifies the affect level for the particular root 
concerned. When a negation prefix is encountered, the original affect code 
is subtracted from 4 to reverse the affect level. This reversal is also . 
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SUBJECT CODE ■ 



OOO^MFT 



••• PROTOCOL REDUCTION SUMMARY ••• 



.♦ SELF DESCRIPTION • ‘ s 

1 I AM •OISMEADTEN* • R a (UNIQUE) T ■ . A a 0 

17 1 AM NOT *OPEMMINDFD* R a OPEN T ■ COG' A a \ 

4 OTHER PEOPLE THINK THAT I AM •TRRESPONSI • R a IRRF.SPONSI T ■ P£RF A a 1 

24 OTHER PEOPLE O0NT KNOW THAT I AM •EMOTIONAL a EMOTION T ■ EMOT A a ? 

' 7 MV BEST QUALITY IS MV ^INTELLIGENT R a INTELLI T ■ COG A a 3 

?0 MY WORST QUALTTY IS MV , •LAZINESS* R a LAZ . T ■ PERF A • 1 

15 MY RODY IS •GOOD* R a QOOO ' T ■ PERF A a 3 

. .. ?B MY FMOTIONS ARE R ■ CONTROL V T ■ PERF A « 3 

30 MY MINI) IS •GOOD* R a good T ■ PERF A ■ 3 

3 MV WORK HAS BFEN •EASY* R ■ EASY , T ■ PEMF A a 3 

' ,.*« • • FUTURE ORIENTATION AND GOALS • 

? I SEE *HAPPlNFSS* WHEN I LOOK AHEAD a HAPP T • RMOT A a 3 

19 TEN YFARS FROM NOW I WILL RE •MARRIED* R a MARP !* T a ROLE A ■ ? 

?3 I AM •CERTAIN# AROlJT WHO I AM AND WHERE I AM GOING 0 a CERTAIN T » COG A a 3 

3f* RFAL-LIFE STORIES USUALLY HAVE *REAL # ENDINGS R ■ REAL T a PHYS A a 3 

lft THE MOST IMPORTANT THING IN LIFE IS ■ HAPP T a EMOT A a 3 

?6 WHAT I WANT MOST IS •HAPPINESS* R • HApp T ■ EMOT A a 3 

9 *WORK* LEADS TO SUCCESS R a WORK T ■ PERF A a 3 

?9 »W0RK* LEADS TO SUCCESS R a WORK T ■ PERE A * 3 

12 SECRETLY* I OFTEN DREAM ABOUT “ANXIETIES* R a ANXI T ■ EMOT A a \ 

• STRESSORS AND REACTIONS • 

ft I OONT LIKE TO THINK ABOUT *STUDVIN8* R a STUOY T ■ COG A a ? 

10 *HOMOSEXUAL* MAKES ME ANGRY R a (UNIQUE) T ■ A a 0 

' 14 I AM AFRAID OF R a FAIL T ■ PERF A a 1 

3? *HOME* MAKES ME FEEL SAD a HOME T ■ ROLF A a ? 

R DARKNESS IS *0000* R a QOOO T ■ PERF A a 3 > 

/ , ; 22 WHEN AN ANIMAL IS WILD. IT IS *UNCONTPOLL* R a -CONTROL 1 ; ' T * PERF A a 1 • 

: * '< 35 I *IMPROVF* WHEN PUT IJmoEP PRESSURE R a IMPROV T ■ PERF A a 3 

• ‘ 33 1 *DRINK* WHEN 1 1 LET DO ■ QRTNK T ■ PHYS ( .A a ? 

, • .;?•••;. '• •! 1 

' ' * * SOCIAL ATTITUDES AND INTERACTION • ‘ 

■ .‘.L t .1 ' 

5 MEN OFTEN *OESIPE* R a OESIR T ■ PERF A a 2 

• IB WOMFN OFTEN •SATISFY* MEN R a SATISF T * EMOT A ■ 3 

' ?\ I HAVE «MANy* R a MANY T * PHYS A a ? 

: 34 OTHER PEOPLE MAKE ME *EXAGGERATE* R a (UNIQUE) T ■ ■ A a 0 

' 11 THE AVERAGE PERSON IS *St)CCESSFUL* R a SUCCE T ■ PERE A a 3 

’ 25 MOST MEN ARE *STROnG* p a STRONG T ■ PHYS A a, 3 

. 13 MOST WOMEN ARF *BlTCHES* R a RtTCH. T • SOC . A « 1 

l 31 MY MOTHER IS •SWEET* R a SWEET T ■ SOC A a 3 A 

V; ?7 MY FATHER IS *GOOD« p a GOOD ' T ■ PEHF A a 3 

-y. 4 .*., ■ ■ f . /■*.'' ‘ : 1 '* < ■' f ■ v,. . . - t V 

, .’ i ' *'■ T ; ' ",.i- ' '■ ‘ . . ■ • ■ ■ ■■ (' \ :'\V ' ■ <-)*’ ■; ■ *.’• •'*' 

• i ‘ * ' ■ . .■ . / '* l *. t , ■ I ' . ■ ■ - ; . ' ' ■' . " \ , ■» ■ ' •• ’ ■ ' 1* ■ * ' , . - ' i ■■ •, ■ 

;A; .kvi • . .... ■ •' •• «■;. . *..••• . *. ■■ 

\ I'l "i .» %1 4 1 % i /.■*.' 1 * | 9 t i - 4 ’ ' 1 ' ' , ? « > , t * \ ' -t ’ / m m ' i ^ 

J-.'i V • i'V • ••••: ' \ .:!■ .‘rV^ 

■' .* • V ’• v. ^ ^ r 

. M. ,’ * * - • ■ ■- -V- „ • . ■’ * - ■. ’ • 1 'J ■ • 

• • ••A .)>• • .•••■' : \ . • . ... • 

"a ; ; ^ - ; ' v'-.',' 1 . 

jiE \ : 1 ■' 1 ' ; V . ■ v.-; ■■ 

< • • • , ■ ' ■■■■•- : : • • 4 - V 







SUBJECT CODE ■ ftOOPMFT 

* PROTOCOL CHARACTERISTICS • 

o# plan* responses, 

0, PROPFR-NAMF RFSPOMSES. 

9*00 POPUl AR RESPONSES. 

3.00 IJNTOUE rfspo> ses. 

6.00 repfatfo responses, 

7. OR » AVERAGE RESPONSE LENGTH. 



■' • AFFECT ANALYSIS • 

’ 7.00 NEGATIVE RESPONSES. 

i • * * 

7.00 NEUTRAL responses. 

19.00 POSITIVE RESPONSES. 

2.36 ■ AVERAGE AFFECT LEVEL. 



■ ' ’ ' I*; ' * ’ ‘ ‘ •, 

* STEM-SET ANALYSTS • 

SELF DESCRIPTION A ■ 2.22 

FUTURE ORIENTATION AND GOALS A • 2,67 



1 v 




STRESSORS AMO REACTIONS A ■ 2,00 

SOCIAI ATTITUDES AMD INTERACTION A • 2.50 



* THEMATIC 
PHYSICAL 
ROLES 
EMOTIONAL 
SOCIAL 
COGNITIVE 
PERFORMANCE 



ANALYSTS 

F ■ 4.00 

F ■ ?.00 

F ■ 4.00 

rF « ?.00 ! 

F ■ 4.00 

1 F ■ 15.00 , 



i . •• 

A ■ 2.50 
A ■ 2.00 
A ■ 2.5o 
A ■ 2.00 
A ■ 2.25 > 
A ■ 2.40 








always applied to the response to stem 17 (first stem group) . The theme 

and affect codes form the primary data for construction of most of the 26 

scores on the second page of the report. An accidental, but interesting, 

artifact of the 36-item form appears in group two with stems 9 and 29, 

which are identical. They do not always elicit identical responses, however 

The second page of the report contains the 26 sunmary scores 

listed earlier, with appropriate labeling. Although one could employ these 

scores in studies comparing criterion groups such as males and females — 

as we will see later in this paper — direct interpretation of the scores 

for a single subject would be dangerous at best, without some sort of 

normative data. This particular subject seems to have used a large number 

of "performance" terms, for instance, but it is impossible for us to say 

at this point how unusual such a score might be. Even with normative data 

\ 

available for each separate variable, we will probably design our later 
report-writing routines so as to qualify statements based on one score in 
situations where other scores (such as the blank count) are unusual. Thus, 
even after obtaining normative tables for each of these 26 variables, a 
great deal of further work will be necessary to produce, a sufficiently 
sophisticated report-generating program. 

Male-Female Differences 

The 26 basic scores described earlier were developed for each of 
the 2321 protocols of the freshman sample, and the 1362 males were compared 
to the 959 females on each of these variables. Table 1 contains the re- 
sults of the analyses of variance. The reduced N for some variables was 
necessitated by the zero affect-level scares which resulted when a subject 
gave no responses in the particular category being considered. 
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These results clearly indicate that females use more popular 
response words, and fewer unique response words, than do males. Females 
also repeat response words less often within their protocols, and tend to 
use longer words as responses. Female subjects as a group use more negative- 
affect responses than do males. There is a tendency for males to use more 
neutral responses, but there is no sex difference in the use of positive- 
affect words. These results are in essential agreement with those reported 
in an earlier memorandum (Veldman, 1965). 

The breakdown of data in the present system permits us to explore 
further the matter of sex differences in response to various types of stems. 
It is quite clear Irom Table 1 that the females' use of negative response 
words occurs most heavily on those stems which present stressors or stress 
reactions. There is also a clear tendency for the female sample to use more 
negative words for self -descript ion, although this difference is not as 
strong. 

When the response words are classified according to the six-theme 
system, significant sex differences in frequency of usage occur for every 
one of the six categories. Males use the Physical, Cognitive, and Perform- 
ance themes more often, while females use the Role, Dnotional, and Social 
themes more frequently. 

The last six variables register the average affect levels of the 
words in each of the six thematic categories. Subjects who used no words 
in a particular category were excluded from the corresponding analysis of 
variance. No significant differences appeared for the Social or Performance 
themes. Males responded with more positive affect in the Physical, Brotion- 
al, and Cognitive thematic categories . while females gave more positive Role 
theme responses. 
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Although many of the assignments of response words to thematic 
categories are open to question, it is quite clear that males and females 
respond to the sentence completion instrument in distinctively different 
ways. Beyond the strong suggestion that females respond with generally 
greater verbal facility to this task, patterns of response themes appear 
which are clearly sex-role appropriate. The fact that females use more 
words with negative affective loadings is not easily explained, however, 
and is contrary to what might have been predicted from the stereotypic 
college coed personality pattern. That these negative words appear most 
frequently in response to stems presenting stressors and stress reactions 
only partially explains the phenomenon, since the same sex difference occurs 
with self-descriptive items. 

These strong sex differences in response tendencies dictate the 
development of separate normative tables for males and females for use with 
future report-writing procedures. 



Table 1. Males vs. Females Analyses of Variance 





Variable 


Ns 


Males 


Females 


F 


P 


1 . 


Number of Blanks 


(1362, 959) 


.64 


.50 


1.74 


.1842 


2. 


Number of Proper Names 


(1362, 959) 


.11 


.12 


.14 


.7084 


3. 


Number of Populars 


(1362, 959) 


6.48 


7.83 


113.63 


<.00005 


4. 


Number of Uniques 


(1362, 959) 


1.14 


.88 


19.30 


.0001 


5. 


Number of Repetitions 


(1362, 959) 


4.17 


3.65 


30.08 


<.00005 


6. 


Average Response Length 


(1362, 959) 


6.70 


6.78 


9.43 


.0026 


7. 


Number of Negative Affect 


(1362, 959) 


10.64 


11.36 


23.95 


<.00005 


8. 


Number of Neutral Affect 


(1362, 959) 


7.55 


7.37 


2.35 


.1210 


9. 


Number of Positive Affect 


(1362, 959) 


16.03 


15.89 


.72 


.5974 


10. 


Average Affect Level 


(1362, 959) 


2.16 


2.13 


11.08 


.0013 


11. 


Self-Description Affect 


(1362, 958) 


2.16 


2.12 


8.67 


.0036 


12. 


Future-Goals Affect 


(1362, 959) 


2.50 


2.49 


1.24 


.2644 


13. 


Stress Reactions Affect 


(1362, 959) 


1.61 


1.54 


30.36 


<.00005 


14. 


Social Attitudes Affect 


(1362, 959) 


2.28 


2.30 


1.05 


.3073 


15. 


Physical Theme Count 


(1362, 959) 


5.42 


5.08 


12.88 


.0006 


16. 


Role Theme Count 


(1362, 959) 


2.69 


2.92 


9.76 


.0022 


17. 


Emotional Theme Count 


(1362, 959) 


7.31 


9.21 


239.99 


<.00005 


18. 


Social Theme Count 


(1362, 959) 


4.13 


4.72 


40.22 


<.00005 


19. 


Cognitive Theme Count 


(1362, 959) 


4.30 


3.98 


13.21 


.0006 


20. 


Performance Theme Count 


(1362, 959) 


10.38 


8.71 


159.53 


<.00005 



1.92 10.04 .0020 

1.95 52.32 <.00005 

2.29 14.81 .0003 

1.99 .38 .5437 

2.07 12.07 .0009 

2.27 .14 .7107 



21. Physical Theme Affect 


(1359, 953) 


1.96 


22. Role Theme Affect 


(1242, 916) 


1.88 


23. Eonotional Theme Affect . 


(1360, 958) 


2.35 


24. Social Theme Affect 


(1329, 949) 


1.97 


25. Cognitive Theme Affect 


(1348, 943) 


2.15 


26. Performance Theme Affect 


(1362, 958) 


2.27 



Intercorrelation of Scores 



Separate 26 x 26 correlation matrices were computed for the male 
and female subsamples. They will not be reproduced here, since most of the 
coefficients reflect only the nature of the response categorization system. 
Blank and unique responses were excluded from this system, however, and the 
^^ici"tionships between these counts and the various affect-score averages are 
shown below: 

Blank Count Unique Count 





Males 


Females 


Males 


Females 


All Responses 


.01 


.01 


-.12 


-.09 


Self Stems 


-.02 


.03 


-.05 


-.03 


Future Stems 


.01 


.04 


-.15 


-.13; 


Stress Stems 


.05 


.00 


.11 


.14 


Social Stems 


-.02 


- . 05 


-.18 


-.17 


Physical Responses 


.02 


.01 


-.02 


.01 


Role Responses 


-.05 


-.01 


, .01 


-.03 


Bnotional Responses 


.01 . 


-.04 


-.09 


-.05 


Social Responses 


.03 


.10 


-.04 


-.04 • 


Cognitive Responses 


-.03 


.08 


.04 


-.06 


Performance Responses 


.00 


-.07 


-.08 


.01 



It is quite clear from these data that the giving of blank re- 
sponses is not associated with the affect level of responses to other 
stems in general, nor within any of the stem groups. There appears to be 
a slight tendency for females who give many blanks to also give positive j'V 
social and cognitive responses, but negative performance responses, v " .. 



The tendency to use unique response words seems to be associated 
slightly with the use of negative-affect words on other steins generally. 
This trend is especially true for stems concerning future orientation and 
social interaction, but the reverse appears with stems concerning stress . 
stimuli. A high count of uniques tends slightly to be associated with 
lower emotional and performance-response affect with males, but no relation 
ships appear for the female sample. 

The intercorrelations of the affect averages for the four stem : 
groups are shown in the next table. For both sexes, the affect level for 
self -description is associated with that for future orientation and for 
social interaction, but is not related to the affect level for stress- 
stem responses. The social interaction and future orientation affect 
levels are also associated to about the same degree. The lack of relation- 
ship between stress-stem responses and the others is probably due to the 
fact that these stems are constructed to elicit negative reactions. 



Intercorrelation of Affect Averages for Stem Groups 





Self 


Future 


Strets 


■, Social 


'?• ' ’• ' . ■ ■■ ■ 


Self 


— 


.29 


.01 


.24 


Males above diagonal 


Future 


.29 




-.13 


.21 


Females below diagonal 


Stress 


.00 


-.03 


— 


-.07 




Social 


.18 


.22 


-.13 







The next table of intercorrelations reflects the associations 
among the affect levels of the various response categories. In general, these 
coefficients are surprisingly low. The strongest relationships are between 
emotional responses on the one hand, and cognitive and performance responses 



on the other. These protocols certainly do not seem to be homogeneous 
in mood. To seme extent these coefficients suggest that the present : 
scoring system is not sensitive enough to the nature of the particular 
stems eliciting the response words. For example, 

" Stupidity makes m6 angry. " 
and "My best quality is my intelligence ." 

yield opposite affect scores in the cognitive response category, yet they 
certainly are consistent with each other. Such internal contradictions 

would reduce any cross-category relationships. 

Inter-correlation of Affect Avera ges for Response Categ ories 






Physical 


Role 


Emotional 


Social 


Cognitive 


Performance 


Physical 


— 


-.02 


.07 


.09 


.05 


.01 


Role 


.00 


— 


-.02 


.00 


-.05 


-.03 \ 


Emotional 


.09 


-.06 


mmmm § 


.10 


.11 


.. ' v: .13 / 


Social 


.01 


.01 


.08 




.08 


.06 


Cognitive 


.07 


-.03 


.16 


.05 




'X'-;". 13/ 


Perfornonce 


.03 


-.03 


.14 


-.01 


.09 
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Discussion ' . !! 

. ; s , The system described in this progress report has a number of V- 

weaknesses which are known at this time; others will undoubtedly 0 c ' 

apparent as work continues to improve the flexibility and precision of ; 

the scoring procedures. It may be worthwhile to specify some of these 
: directions for efforts to improve the present system. ; 

0ne weakness is the relatively large number of roots retained 
which contain coimon prefixes not cut out by the present ROOT routine. . ' ' 
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An earlier memorandum described a more complex root-cutting procedure 
(Veldman, 1965), and we will probably modify the present system in that 
direction in order to reduce the size of the root dictionary and increase 
its range of applicability. We also may wish to consider the practicality 
of hand-screening future data to reduce multi-word responses to key words 
prior to punching. 

The usefulness of. a system like the one described here, which 
reduces the responses to only six thematic categories and the stems to 
four groups, is open to serious question as a general approach to the goal 
of developing a scoring system for the OWSC data. The primary question is 
one of the adaptability of the system to the variety of purposes to which 
sentence completion data may be applied in various research situations. 

It has been suggested that the present system is too concise to allow 
users to get meaningful answers to particular questions they might hope 
to answer with the instrument. Suppose one wished to know the subject’s 
attitude toward his parents. About the closest score to this area would 
be' the Role Affect level score — which, of course, registers attitudes i. 
toward a much wider variety of role-objects and institutions. Lack of 
ability to answer specific questions within the system framework is not 
the only weakness; one might also wish to develop broadly-based scores of 
a different nature than those which are yielded by the current system — 
even in its 72-cell form. 

One way of building a more flexible system will be outlined 
here. We intend to implement this approach with the Freshmen Study data 
in order to test its feasibility. The system will operate as outlined in 
the present report up to the semantic stage of response equivalencing — 
with perhaps an increase in the prefix-chopping power of the ROOT routine. 
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Instead of the rather extreme reduction to six themes at the pragmatic . 
level, however, we will employ somewhat fewer than 100 thematic categories 
of synonymic root-sets. Each synonymic set will also be coded for "dimen- 
sionality." The notion here is that some thematic categories can be or- 
ganized along an affect dimension. For instance, "love-hate" might form 
a thematic category, within which LOVE and ADORE might be coded "5" for \ 
dimensionality, while LIKE would be coded "4", HATE would be coded "1", 
etc. Other thematic categories would also contain terms coded for "dimen- y 
sionality j" but the dimension would not be affective in nature. For in- 
stance, in the thematic category "parents," FATHER would be coded "4", and 
MOTHER would be coded "2"; the numerical codes would only signify the 
dichotomous nature of the theme. Ambiguous terms like PARENT would be 
coded "3". Still other themes would be dimensionalized with a more general 
evaluative loading, such as "size*" 

Use of such a less- than-1 00-theme system, with further differ- • 
entiation by a one-digit dimensionality code , would permit the score de- 
signer considerable versatility. When we then add to the system a two- ; 

• • • ■ •' 

digit code for the particular sentence-completion stem concerned , each ' 
protocol can be described in terms of 36 five-digit codes. Scores can be i 

• \'.y 

defined as sums of dimensionality codes for particular thematic categories 
and/or particular stems, or in any of a wide variety of other ways. 

To make such a data-reduction system most useful to psychologists, 
it would be necessary to develop a computer program which would accept 
formulas for specific scores, written in a concise notation. Seme "stand- 
ard" formulas could be developed initially as a general-purpose scoring ^ 
system, but the organization of the data-reduction scheme and the program 
for translating and implementing score formulas would enable rapid and 
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economical scoring of any set of protocols for an almost infinite variety 
of special purposes. 

Finally, we may consider the advisability of designing a new 
OWSC form to recommend in all future research situations. Accumulated 
experience with the data provided both by the 90 and 36-item versions ' / 
strongly suggests that it wculd be best to limit the stem formats to two 
more-or-less standard arrangements. The most straightforward type of stem 
presents an object and asks for a descriptor. For instance, "My father is 

•" ^ reverse arrangement has also been found useful: "I am proud 

m y •" Stem formats other than these within a protocol complicate 

the root-reduction system, and seldom contribute data which cannot be ob- 
tained from the formats shown above . To increase the usefulness of the 
OWSC as a screening instrument, we should probably construct stems which 
would be useful in a wide variety of research and individual-assessment 
situations, with high school, college and adult subjects, and yet short 

. ■ ■ - ■ . .1 , . 

enough (perhaps 20 - 30 stems) to be suitable for use in mass testing 
programs. . \ 
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